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Abstract 

o 

^^f-^ We consider the problem of calibrating a compressed sensing mea- 

surement system under the assumption that the decalibration consists 
in unknown gains on each measure. We focus on blind calibration, us- 
ing measures performed on a few unknown (but sparse) signals. A naive 
formulation of this blind calibration problem, using ii minimization, is 
^ fH reminiscent of blind source separation and dictionary learning, which are 

4--^ known to be highly non-convex and riddled with local minima. In the 

2 considered context, we show that in fact this formulation can be exactly 

y expressed as a convex optimization problem, and can be solved using off- 

the-shelf algorithms. Numerical simulations demonstrate the effectiveness 
^ of the approach even for highly uncalibrated measures, when a sufficient 

^ number of (unknown, but sparse) calibrating signals is provided. We ob- 

QQ serve that the success/failure of the approach seems to obey sharp phase 

transitions. 

(N 

^ 1 Introduction 

Linear inverse problems are ubiquitous in signal and image processing, where 
i-H they are used to estimate an unknown signal xq G or from noisy linear 

^ measurements: 

y :=Mxo + nGM^ or C^. 



^ When m < N this well-known under-determined problem admits infinitely many 

^ solutions, but if xq is sparse enough it can be estimated accurately using sparse 

regularization. Among many other techniques, li regularization has recently 
become quite popular and consists in solving, e.g., 

xi := argmin ||x||i s.t. ||y - Mx||2 < e 

X 

for a well chosen e. When the measurement matrix M is perfectly known, 
this approach is known to perform well, and a well-established body of work 
characterizes its performance guarantees for the recovery of vectors xq that are 
sufficiently well approximated by highly sparse vectors (see, e.g. [li|2l[3j). Such 
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sparse linear regression problems occur in many practical scenarii where the 
measurement matrix M is either dictated by the physics of the measurement 
system, or designed to have favorable properties with respect to the recovery of 
sparse vectors: this is the now famous compressed sensing (CS) scenario (see, 
e.g. [4l|5j[6]), where M voluntarily reduces dimension, exploiting the sparsity of 
xo to capture it with fewer measurements than Nyquist sampling would require. 

1.1 The decalibration issue 

In practical situations, the true measurement system M is not perfectly known: 
it may only have been modeled; or it may have been measured through a cali- 
bration process, but the physical conditions of the system (such as temperature) 
may have drifted since this calibration. 

Exploiting sparse regularization with an inaccurate estimate M of the true 
measurement system M is likely to hurt the reconstruction performance [7]. It 
is believed to be one of the reasons limiting the observed performance of several 
compressed sensing devices. To address this problem, the most standard existing 
approaches are: 

1. To ignore the problem. 

2. To consider de-calibration as noise 0: y ~ Mxo + (M — M)xo. This 
leads to solving 

xi := argmin ||x||i s.t. ||y - Mx||2 < e + 7/ 

X 

with T] an estimate of the magnitude of this added noise. 

3. Supervised calibration: using known training signals xi, . . . xi, and the 

corresponding observations = Mx^+n^. Gathering all data in matrices, 
this takes the form Y = MX + N. The matrix M is re-estimated, e.g., as 

M:= argmin II Y-MX|||.. 

M 

1.2 Constraints on cahbration 

It is sometimes useful to constrain the estimated calibration matrix M to belong 
to some family A4 of matrices. For example, it is sometimes known/assumed [8] 
that the unknown M is sparse in a given dictionary of measurement matri- 
ces {M/e}: M « ^^Qf/cM/c, ||a||o small. Supervised calibration can then be 
performed by solving, e.g., the convex relaxation 

min||a||i s.t. || Y - ^ a^M^XllF < e. 

k 

In this paper, we concentrate on a different scenario where the measurement 
matrix is almost known, up to an unknown gain on each measure: that is to 
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say, M = Do Mo where Mo is a perfectly known measurement matrix, and Do 
is an unknown diagonal matrix which i-th entry is a (real or complex) gain di 
applied to the i-th measure of xq associated to the i-th line of Mo- This leads 
to 

M:={M = DMo, D = diag((iO, ^ Vz}. 

Several practical scenarii can be associated with this assumption, for instance 
in the case of a microphone array where the frequency response of each micro- 
phone needs to be individually calibrated |9j . At each frequency, the calibration 
problem amounts to choosing unknown gains. 

1.3 Unsupervised / blind sparse calibration 

In this paper, we are interested in a blind calibration problem, where no known 
training signal can be used. While some training signals have given rise to 
observed measures y^, the training signals themselves are not known. Since one 
could not hope to calibrate the system without some form of knowledge of x^, 
and the final scenario for using M will be sparse reconstruction, we assume that 
the unknown signals x^ are sparse and somehow statistically diverse. In matrix 
form, we have Y = DoMoXo where Y is the mxN (known) observation matrix. 
Do is the square and diagonal matrix of size m for the (unknown) calibration 
coefficients. Mo is the mx N (known) idealized measurement matrix, and Xo is 
the (unknown) x L set of calibration signals upon which the only knowledge 
we have is that they are /c-sparse. 

The objective is to derive a calibration technique that exploits the knowledge 
of Y and Mo, as well as the sparsity of the unknown Xo, to obtain estimates 
D and X, and finally M = DMo. It seems natural to consider the following 
optimization problem, where the £i objective function is intended to promote 
the sparsity of X: 

minllXlli s.t.Y = DMoX. (1) 

D,X " " ^ ^ 

However, this would naively lead to two major issues: 

1. Without further constraint on D and/or X, one can scale D to infinity 
while letting X go to zero, leading to a trivial but uninteresting solution; 
This is usually solved through a normalization constraint D G P where 
the set V is bounded; 

2. Even with appropriate normalizing constraints on D, this seems at first a 
non-convex problem, because of the bilinear nature of the term DMoX, 
which is linear / convex separately in D and X but not jointly. 

1.4 Relation to previous work 

The above issues are well known since they are encountered in blind source 
separation [10 and dictionary learning JTJ [12]. In a way, the unsupervised 
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calibration problem can be seen as a simplified instance of dictionary learning, 
where the general problem has been expressed, e.g, as [131 ttH HSl US] 

minllXlli s.t.Y = 

where the minimum is over $ G C for a well-chosen bounded set C (typically, the 
set of matrices with unit norm columns, or oblique manifold)^ but the dictionary 
$ is not restricted to be diagonal. 

The present issue is also related to the basis mismatch problem [17], where 
the signals are truly sparse in a basis that is slightly different from the chosen 
representation basis (for instance with a parametric dictionary using a grid of 
quantized parameters); or to the perturbed CS problem ^HllTQ], where multi- 
plicative noise limits the effectiveness of CS. In both cases, the "noise" on the 
measurement matrix is different for every vector, whereas in the decalibration 
case considered here, this (unknown) basis is shared by all vectors. 

1.5 Contributions 

The main contribution of this paper is (Section [2| to show that, under proper 
parametrization and normalization, the considered unsupervised calibration prob- 
lem ([T]) can actually be exactly expressed as a convex optimization problem. Nu- 
merically solving this problem is straightforward using off-the-shelf algorithms. 
In numerical experiments (Section |3|, we demonstrate the effectiveness of this 
approach even for highly uncalibrated measurements, whenever a sufficient num- 
ber of (unknown but sparse) calibrating signals is provided. Remarkably, we 
observe that the success/failure of this problem obeys some sharp phase tran- 
sitions, generalizing to the uncalibrated case the phase transitions studied by 
Donoho and Tanner [2^ . 

2 Proposed approach 

As noticed above, the naive formulation of the problem is non-convex, even with 
a (convex) normalization constraint. 

2.1 Convex formulation 

To provide a convex formulation we propose to reparameterize the problem. 
Using the simple assumption that the unknown gains di are nonzero, we can 
write the constraint Y = DMqX as AY = MqX, where A = D"^ = diag(5i). 

2.2 Normalization constraint 

With this new parametrization, the constraint is convex in the pair (A,X). 
However, it is trivially satisfied for the pair (0,0). To avoid this trivial solution 
we introduce a convex normalization constraint Tr( A) = 5i = m. Therefore, 
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Figure 1: Empirical phase-transitions with no cahbration (top) and proposed 
cahbration (bottom). Overwhelming success (white) or failure (black) is dis- 
played as a function of dimension parameters J := mjN (abscissa) and p := k/m 
(ordinate). From left to right: increasing values of the number L of training 
signals and decalibration amplitude <j. The thin line is the asymptotic Donoho- 
Tanner phase transition curve for known Gaussian measurement matrix. Crosses 
on the bottom- left figure indicate choices of ( (5, p ) for the experiments on Fig. [2] 



the constraint set is P := {D = diag((i^), d-^= m}. Note that many other 
alternatives could be considered such as (5i = 1. 

2.3 Proposed blind calibration approach 

We end up proposing the following unsupervised calibration approach: given the 
collection of observed measures Y and the model of the measurement system 
Mo, we estimate the inverse of the calibration gains A and the training signals 
X as: 

(Xcai,A) :=argmin||X||i s.t. AY = MqX, Tr(A) = m. (2) 
This is a convex problem. 



3 Experimental results 
3.1 Considered techniques 

We compare two techniques. The proposed approach ([2|, and an approach 
which ignores decalibration 

Xuncai := argmin ||X||i s.t. Y = MqX. 



Both convex problems are solved using the cvx toolbox [21]. 



3.2 Data generation 

The data was generated by drawing i.i.d. random /c-sparse vectors with a k- 
sparse support chosen uniformly at random and i.i.d. Gaussian nonzero entries. 
The ideahzed measurement matrix Mq was drawn from the Gaussian ensemble 
of size m X N with N = 100. Real positive decalibration coefficients were gen- 
erated using Do = diag(exp(A/'(0, cr^))), where a is the parameter governing the 
amplitude of decalibration. Hence, a given value of a leads to a decalibration 
offset in decibels with zero mean and standard deviation ±20cr/ln(10) ^ ±8.7a 
dB. The considered measures were Y = DoMqX, where X was a concatenation 
of L i.i.d. /c-sparse signals. Experiments were conducted for various configura- 
tions of S := m/N, p := k/m^ L and a to measure how accurately the signals X 
were recovered with the different approaches. Recovery was considered success- 
ful if the normalized cross-correlation between original and estimated signals 
is above 99.5 % (note that, due to the global scaling invariance in A and X 
mentioned in the introduction, a distance measure between X and X would 
not be a relevant indication of success). All the empirical phase transition di- 
agrams described below report the number of successful estimations, when the 
experiment was repeated over 50 random draws. 

3.3 Can CS still succeed under calibration errors ? 

The first study dealt with estimating how much classical CS was robust to cali- 
bration errors. To do this, we ran numerical experiments and plotted empirical 
Donoho- Tanner phase transition ^0] (in the framework presented above). Dif- 
ferent de-calibration levels a and number L of calibration signals have been 
tested. 

The results can be seen on the top line of Figjlj for small values of de- 
calibration (cr = .01, a = .0316), the transition curve is barely modified : CS 
seems indeed robust to small calibration errors. However, as the de-calibration 
increases (<j = .1, i.e. the decalibration error was of the order ±.9 dB), then 
the region where CS succeeds drastically shrinks, and eventually disappears at 
even larger de-calibration (cr = .316 and above). 

The second line of fig. [l] shows the result of CS under the blind calibra- 
tion procedure introduced above. For a large number of noise levels where the 
un-calibrated experiment failed, it now succeeds even with a relatively small 
number L of training samples (for instance, with L = 21 training samples, it 
still succeeds at a = 1, i.e., a decalibration error of the order ±8.7 dB). Inter- 
estingly, for very small values of de-calibration (a = .01 and below) and too 
few training samples, it is better not to perform the blind calibration, as it 
introduces too many degrees of freedom. 

3.4 Choosing the number of training samples 

In this section we wish to determine how many training samples L are needed 
as a function of the "noise" level a. We picked different values of (J, p) where 
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Figure 2: Experiments at fixed as a function of the number L of training 

signals and the decalibration amplitude a. "Uncalibrated" and "calibrated" 
refer to the standard and proposed algorithms, respectively. White indicates 
that the algorithm succeeds in recovering the solution, and black indicates a 
failure. 



ideal CS works, and plotted on fig|2] the rate of success as a function of de- 
calibration level <j and number of training samples L, both in the un-calibrated 
and calibrated cases. Remarkably, these obey sharp corner-like phase transi- 
tion. Calibration allows CS to work for a increased by an order of magnitude 
(compared to the un-calibrated case), provided L is larger than some threshold. 
These threshold values seem to have some universality, as numerical tests do not 
indicate any clear dependency on the ambient dimension N (results not shown 
here), and sharper transitions are observed for larger values of N. 



4 Conclusion 

In this study, we have experimentally observed that even mild decalibration on 
the measurement matrix can lead to a spectacular failure of standard CS re- 
covery algorithms. This problem is highly relevant for engineering applications, 
as in many cases the observation matrix is not exactly known. The proposed 
blind calibration technique, formulated here as a convex optimization problem, 
is shown to offer a significantly improved robustness to decalibration: when a 
sufficient number of (unknown but sparse) calibration signals are provided, the 
algorithm can now succeed even for much larger calibration errors. Remarkably, 
the success / failure of this new calibrated CS exhibits sharp corner-like phase 
transitions. Being able to compute theoretically the asymptotic boundaries of 
these new transition diagrams is a promising extension of this work, together 
with the extension to more general decalibration cases. More immediate targets 
will be the design of efficient blind calibration algorithms to scale the approach 
and test in on real calibration problems arising in compressed sensing of acoustic 
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fields [9]. 



References 

[1] J. -J. Fuchs, "On sparse representations in arbitrary redundant bases," 
IEEE Trans. Inform. Theory, vol. 50, no. 6, pp. 1341-1344, June 2004. 

[2] Emmanuel Candes, "The restricted isometry property and its implica- 
tions for compressed sensing," Compte Rendus de V Academic dcs Sciences, 
Paris, Series /, vol. 346, pp. 589-592, 2008. 

[3] R.G. Baraniuk, V. Cevher, and M.B. Wakin, "Low-dimensional models for 
dimensionality reduction and signal recovery: A geometric perspective," 
Proceedings of the IEEE, vol. 98, no. 6, pp. 959 -971, June 2010. 

[4] David L. Donoho, "Compressed sensing," IEEE Trans. Inform. Theory, 
vol. 52, no. 4, pp. 1289-1306, 2006. 

[5] E. J. Candes, J. Romberg, and Terence Tao, "Stable signal recovery from 
incomplete and inaccurate measurements," Comm. Pure Appl. Math, vol. 
59, pp. 1207-1223, 2006. 

[6] Richard Baraniuk, "Compressive sensing," IEEE Signal Processing Maga- 
zine, vol. 24, no. 4, pp. 118-121, July 2007. 

[7] M.A. Herman and T. Strohmer, "General deviants: An analysis of per- 
turbations in compressed sensing," Selected Topics in Signal Processing, 
IEEE Journal of vol. 4, no. 2, pp. 342 -349, april 2010. 

[8] G.E. Pfander, H. Rauhut, and J. Tanner, "Identification of matrices having 
a sparse representation," Signal Processing, IEEE Transactions on, vol. 56, 
no. 11, pp. 5376-5388, Nov. 2008. 

[9] Remi Mignot, Laurent Daudet, and Francois Ollivier, "Compressed sensing 
for acoustic response reconstruction: Interpolation of the early part," in 
Applications of Signal Processing to Audio and Acoustics, IEEE Workshop 
on (WASPAA), New York, NY, USA, Oct. 2011. 

[10] M. Zibulevsky, B.A. Pearlmutter, Pau Bofill, and P. Kisilev, "Blind source 
separation by sparse decomposition," in Independent Component Analysis: 
Principles and Practice, S. J. Roberts and R.M. Everson, Eds., pp. 181-208. 
Cambridge Press, 2001. 

[11] Kenneth Kreutz-Delgado, Joseph F. Murray, Bhaskar D. Rao, Kjersti En- 
gan, Te-Won Lee, and Terrence J. Sejnowski, "Dictionary learning algo- 
rithms for sparse representation," Neural Comput., vol. 15, no. 2, pp. 
349-396, 2003. 



8 



[12] M. Aharon, M. Elad, and A.M. Bruckstein, "The K-SVD: An algorithm 
for designing of overcomplete dictionaries for sparse representation," IEEE 
Transactions on Signal Processing^ vol. 54, no. 11, pp. 4311-4322, Nov. 
2006. 

[13] M. Yaghoobi, T. Blumensath, and M. E. Davies, "Regularized dictionary 
learning for sparse approximation," in Proceedings of EUSIPCO^OS, 2008, 
vol. to appear. 

[14] M. Yaghoobi, T. Blumensath, and M.E. Davies, "Dictionary learning for 
sparse approximations with the majorization method," Signal Processing, 
IEEE Transactions on, vol. 57, no. 6, pp. 2178 -2191, June 2009. 

[15] Remi Gribonval and Karin Schnass, "Dictionary identifiability - sparse 
matrix-factorisation via ii minimisation," Information Theory, IEEE 
Transactions on, vol. 56, no. 7, pp. 3523-3539, July 2010. 

[16] Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro, "Online 
learning for matrix factorization and sparse coding," Journal of Machine 
Learning Research, vol. 11, pp. 19-60, 2010. 

[17] Y. Chi, L. Scharf, A. Pezeshki, and R. Calderbank, "Sensitivity to basis 
mismatch in compressed sensing," Signal Processing, IEEE Transactions 
on, , no. 99, pp. 1-1, 2009. 

[18] M.A. Herman and T. Strohmer, "General deviants: An analysis of per- 
turbations in compressed sensing," Selected Topics in Signal Processing, 
IEEE Journal of vol. 4, no. 2, pp. 342-349, 2010. 

[19] H. Zhu, G. Leus, and G. Giannakis, "Sparsity-cognizant total least-squares 
for perturbed compressive sampling," Signal Processing, IEEE Transac- 
tions on, , no. 99, pp. 1-1, 2010. 

[20] David Donoho and Jared Tanner, "Counting faces of randomly-projected 
polytopes when the projection radically lowers dimension," Journal of the 
AMS, vol. 22, no. 1, pp. 1-53, Jan. 2009. 

[21] Michael Grant, Stephen Boyd, and Yinyu Ye, "CVX: Matlab Software for 
Disciplined Convex Programming," Aug. 2008. 



9 



